Skip to content

Pure-julia OpenAPI internals rewrite - #103

Open
quinnj wants to merge 5 commits into
JuliaComputing:mainfrom
quinnj:codex/production-rewrite
Open

Pure-julia OpenAPI internals rewrite#103
quinnj wants to merge 5 commits into
JuliaComputing:mainfrom
quinnj:codex/production-rewrite

Conversation

@quinnj

@quinnj quinnj commented Aug 6, 2026

Copy link
Copy Markdown

Replace the legacy generated-client and server implementation with the normalized OpenAPI 3.0, 3.1, and 3.2 pipeline.

Keep the provisional JSON Schema engine isolated inside OpenAPI. Generate clients against that engine until its API is ready to move upstream.

Keep HTTP optional through an extension. Leave server framework integration to downstream packages such as Servo.

Add adversarial, conformance, external-corpus, runtime HTTP, and JuliaC trim-compilation coverage.

BREAKING CHANGE: The legacy OpenAPI 0.2 API is replaced by the namespaced document and client-generation API.

quinnj and others added 2 commits August 5, 2026 21:55
Replace the legacy generated-client and server implementation with the normalized OpenAPI 3.0, 3.1, and 3.2 pipeline.

Keep the provisional JSON Schema engine isolated inside OpenAPI. Generate clients against that engine until its API is ready to move upstream.

Keep HTTP optional through an extension. Leave server framework integration to downstream packages such as Servo.

Add adversarial, conformance, external-corpus, runtime HTTP, and JuliaC trim-compilation coverage.

BREAKING CHANGE: The legacy OpenAPI 0.2 API is replaced by the namespaced document and client-generation API.
Add OpenAPI.serverplan and OpenAPI.server(source; framework, name, path),
mirroring the plan/client pipeline. Split the generated runtime into a
direction-agnostic common segment plus client and server segments; the server
segment adds the inverse codecs (path/query/cookie style decoders,
form-urlencoded and multipart/form-data request readers, and a
descriptor-driven response encoder) with request-direction schema validation
and structured 400/415 error responses.

Framework glue is dispatched through the new OpenAPI.server_source extension
seam: OpenAPIHTTPExt emits HTTP.Router modules whose register!(router, impl;
path_prefix, middleware) entry point and handler contract match the shape
OpenAPI.jl 0.2.x julia-server users implement stubs against (register alias
included). Server planning rejects what cannot be decoded faithfully:
non-form-data multipart request bodies and operations with more than one
exploded object query or cookie parameter.

Parameter descriptors gain a shape field and media descriptors a fields
element so single-valued exploded arrays decode as arrays; header scalar
error messages are direction-neutral now that both directions share them.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
RFC 3339 requires an offset, but naive timestamps are what most JSON
serializers print, so strict decoding rejected a large share of deployed
APIs. Be liberal on input: a missing offset now means UTC — the same
convention _encode already applies when it stamps naive DateTimes with Z.
Malformed values and partial offsets still raise DecodeError.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@tanmaykm

tanmaykm commented Aug 9, 2026

Copy link
Copy Markdown
Member

Thanks @quinnj . I did some trials with the new client. Majority of the specs that I tried work fine. But here are a few things which I feel should be addressed:

  • The response code declaration in spec seems too strict. Specs sometimes do not document response code when they do not return any data. With the new client, this throws an error.
  • Missing Content-Type on a response would throw exception. The existing client deserializes by status alone. The new client throws UnexpectedContentType. Sloppy servers, which we do encounter in practice, would become client errors.
  • Date time fields do not support time zone.
  • Streaming support is not there

I will also try this out with some more complicated specs, maybe the k8s api spec.

quinnj and others added 2 commits August 9, 2026 11:55
Address tanmaykm's production trial feedback on the rewrite:

- An undocumented 2XX status no longer throws: an empty body returns
  nothing and a payload returns raw bytes. Undocumented error statuses
  still throw ApiError.
- A response with no Content-Type decodes by status alone, as does a
  misreported Content-Type when only one media type is documented for the
  status. UnexpectedContentType is reserved for genuinely ambiguous
  multi-media responses.
- A new datetime = :zoned generation option maps format: date-time to
  TimeZones.ZonedDateTime with offsets preserved end to end; the default
  Dates.DateTime mapping continues to normalize RFC 3339 offsets to UTC.
- A new stream_to::Channel keyword on every generated operation streams
  response bodies incrementally over HTTP.open: consecutive JSON
  documents, JSON lines, RFC 7464 records, text lines, or raw chunks,
  each decoded to the documented response type. The call returns at the
  response head; closing the channel from the consumer aborts the
  transfer.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Content keys that differ only in parameters are separate entries, not
case-insensitive duplicates: the Kubernetes OpenAPI v3 documents pair
application/json with application/json;stream=watch on every list
operation, and the duplicate check previously rejected the whole
document. Compare the full lowercased key instead of the stripped base
type.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@quinnj

quinnj commented Aug 9, 2026

Copy link
Copy Markdown
Author

Thanks for the thorough trial run, @tanmaykm — all four points are addressed as of ac0689d:

Undocumented response codes no longer error. A 2XX status the spec doesn't describe now succeeds: an empty body returns nothing, a non-empty body returns the raw bytes. Undocumented error statuses still throw ApiError (with the raw body attached) so failures stay visible.

Missing/misreported Content-Type falls back to decoding by status. When a response has no Content-Type, or misreports it while only one media type is documented for that status, the client decodes with the documented media type — the legacy "deserialize by status alone" behavior. UnexpectedContentType is now reserved for the genuinely ambiguous case: several documented media types and a header that matches none of them.

Time zones. Offsets like 2020-01-02T03:04:05+05:30 parse in the default Dates.DateTime mapping by normalizing to UTC (and zone-less date-times are accepted as UTC). For preserved offsets, generate with OpenAPI.client(doc; datetime = :zoned): format: date-time fields then map to TimeZones.ZonedDateTime with RFC 3339 round-tripping, matching the legacy client's ZonedDateTime behavior. The default stays DateTime so plain clients keep no TimeZones dependency (and stay juliac --trim friendly); the option is a one-liner where zone fidelity matters.

Streaming responses. Every generated operation now accepts stream_to = Channel(n). The call returns at the response head and a background task decodes items onto the channel: application/json bodies split into consecutive JSON documents each decoded against the documented schema (the k8s watch convention), JSON Lines/NDJSON decode per line to the array's element type, JSON text sequences split on RFC 7464 records, text/* yields lines, and other media yield raw chunks. Decode/validation failures close the channel with the error, error statuses throw ApiError with the buffered body, and closing the channel from the consumer side aborts the transfer (connection torn down, not leaked).

events = Channel{Any}(16)
K8sClient.watch_core_v1_namespaced_pod(...; stream_to = events)
for event in events
    ...
end

Test coverage added for all of the above, including a raw chunked-transfer fixture that splits items across wire chunks.

I also pre-flighted the k8s trial you mentioned: the v3 documents pair application/json with application/json;stream=watch on every list operation, and the normalizer was rejecting those as case-insensitive duplicate media types. Parameterized keys are now kept distinct, and the core api/v1 document (112 paths, all the watch operations) generates and compiles cleanly. Would still very much appreciate your run against the rest of the k8s groups.

[update prompted and reviewed by quinnj, posted by claude]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants